Repository navigation
perf(cpp): reuse one output TsBlock across the devices of a scan - #1097
Merged
Merged
Conversation
A device-ordered table scan creates a SingleDeviceTsBlockReader per device, and each of them allocated its own output TsBlock: one Vector (and one column buffer) per projected column, faulted in again for every device. The layout is the same for every device of a table, so the allocation only has to happen once. DeviceOrderedTsBlockReader now owns a ReusableTsBlock and hands it to each per-device reader, which passes it down to the multi-value aligned column context; the series scan iterator asks it for the block through a factory instead of allocating one. The block is rebuilt only when the row budget, the column count or a column type changes, and a renamed column updates the tuple description in place. Every acquire is followed by reset(), so each device starts from an empty block. The returned block stays owned by the reader and is reused across devices, as TableResultSet already consumes one block before requesting the next. Reading a 200-column table over 30 devices of 20 rows takes 68.0 ms instead of 71.9 ms per query (mean of 3 runs; the gain grows with the column count, but per-device index loading still dominates the fixed cost).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1097 +/- ##
===========================================
+ Coverage 66.51% 66.54% +0.02%
===========================================
Files 778 778
Lines 56577 56623 +46
Branches 9105 9116 +11
===========================================
+ Hits 37633 37680 +47
+ Misses 17291 17287 -4
- Partials 1653 1656 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
5 open issues
set_reusable_block()is called before the null-check. Even though standardnewthrows on OOM,… · Newdescis passed as a non-const reference, but this API is conceptually describing a required… · NewConsecutiveDeviceScansShareOneBlockverifies \"the right values\" across… · NewTsBlocklayout includes two columns (timeand the… · New
What changed in this PR
Improves performance of device-ordered table scans by reusing a single output TsBlock (and its column buffers) across devices when the layout is identical, avoiding repeated allocations and page-faulting.
Changes:
- Add
ReusableTsBlockand thread it through device-ordered readers to share one output block across per-device scans. - Add an optional
TsBlockFactoryhook toTsFileSeriesScanIteratorto reuse/allocate blocks based on computed layout. - Add new GTest coverage validating reuse, rename-without-rebuild, and rebuild-on-layout-change behaviors.
| File | Description |
|---|---|
| cpp/test/reader/block/reusable_tsblock_test.cc | Adds tests for ReusableTsBlock layout reuse and for sharing one block across consecutive device scans. |
| cpp/src/reader/tsfile_series_scan_iterator.h | Introduces TsBlockFactory API to optionally reuse output blocks. |
| cpp/src/reader/tsfile_series_scan_iterator.cc | Implements alloc_or_reuse_tsblock() to build layout and request a reusable block. |
| cpp/src/reader/block/single_device_tsblock_reader.h | Adds ReusableTsBlock and plumbing to pass it into aligned multi-value reading. |
| cpp/src/reader/block/single_device_tsblock_reader.cc | Wires ReusableTsBlock into VectorMeasurementColumnContext via TsFileSeriesScanIterator factory. |
| cpp/src/reader/block/device_ordered_tsblock_reader.h | Stores a ReusableTsBlock in the device-ordered reader to share across devices. |
| cpp/src/reader/block/device_ordered_tsblock_reader.cc | Sets the reusable block on each per-device reader instance. |
Review effort: Lite
Comment on lines
+72
to
75
| current_reader_->set_reusable_block(&reusable_block_); | ||
| if (current_reader_ == nullptr) { | ||
| return common::E_OOM; | ||
| } |
| // block plus all its per-column buffers for every device. The factory is | ||
| // handed the exact layout the block must have and may return a matching | ||
| // block that is reused across series; returning nullptr makes the iterator | ||
| // allocate a new one. The returned block must already be reset(). |
| // block that is reused across series; returning nullptr makes the iterator | ||
| // allocate a new one. The returned block must already be reset(). | ||
| using TsBlockFactory = common::TsBlock* (*)(void* ctx, | ||
| common::TupleDesc& desc, |
Comment on lines
+188
to
+194
| char* value = nullptr; | ||
| uint32_t len = 0; | ||
| EXPECT_EQ(context.get_current_value(value, len), common::E_OK); | ||
| int64_t v = 0; | ||
| std::memcpy(&v, value, sizeof(v)); | ||
| EXPECT_EQ(v, time - kBaseTime); | ||
| times.push_back(time); |
| for (int i = 0; i < 5; i++) { | ||
| ASSERT_TRUE(rows.add_row()); | ||
| int64_t t = i; | ||
| rows.append(0, reinterpret_cast<char*>(&t), sizeof(t)); |
Copilot stopped reviewing on behalf of
ColinLeeo due to an error
November 12, 2026 17:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A device-ordered table scan (
TableQueryOrdering::DEVICE) creates aSingleDeviceTsBlockReaderper device. Each of them allocated its own outputTsBlock— oneVectorand one column buffer per projected column — and faulted those buffers in again for the next device, although every device of a table has the same layout.Changes
DeviceOrderedTsBlockReaderowns aReusableTsBlockand hands it to the per-device reader (set_reusable_block()), which passes it to the multi-value aligned column context.TsFileSeriesScanIteratorgained an optional block factory (set_tsblock_factory());alloc_or_reuse_tsblock()builds the layout it needs (time + one column per value measurement, from the aligned chunk headers) and asks the factory for a block instead of allocating one. A null return falls back to the old allocation.ReusableTsBlock::acquire()keeps the block when the row budget, the column count and the column types match, refreshes column names in place, and rebuilds otherwise. Every acquire is followed byreset().The returned block stays owned by the reader and is reused across devices;
TableResultSetalready consumes one block completely before asking for the next, so the reader's contract is unchanged for its only caller.Tests
New
cpp/test/reader/block/reusable_tsblock_test.cc:ReusesTheBlockForTheSameLayout— same layout returns the same block; a reset block starts empty.RenamedColumnKeepsTheBlock— a name-only change does not rebuild.RebuildsOnADifferentLayout— a different row budget, column type or column count (andrelease()) builds a new block.ConsecutiveDeviceScansShareOneBlock— three device scans of a real file share one block (TEST_build_count() == 1), each returning all 40 rows with the right values across several TsBlocks.Full
TsFile_Testsuite passes (including the existing multi-device table query tests);mvn -P with-cpp spotless:checkpasses.Benchmark
Table query over 200 INT64 columns, 30 devices x 20 rows, 8 iterations per run, release build on macOS, mean of 3 runs:
The win is the avoided per-device
TsBlockallocation and it scales with the column count, but per-device index loading still dominates the fixed cost of a device-ordered scan (see #1095 for that part).